Human Intent Prediction Using Markov Decision Processes

This paper describes a system for modeling human task-level intent through the use of Markov Decision Processes (MDPs). To maintain safety and efficiency during physicallyproximal human-robot collaboration, it is necessary for both human and robot to communicate or otherwise deconflict physical actions. Human-state aware robot intelligence is necessary to facilitate this. However, physical action deconfliction without explicit communication requires a robot to estimate a human (or robotic) companion’s current action(s) and goal priorities, and then use this information to predict their intended future action sequence. Models tailored to a particular human can also enable online human intent prediction. We call the former a ‘simulated human’ model – one that is non-specific and generalized to statistical norms of human reaction obtained from human subject testing. The latter we call a ‘human matching’ model – one that attempts to produce the same output as a particular human subject, requiring online learning for improved accuracy. We propose the creation of ‘simulated human’ and ‘human matching’ models in this manuscript as a means for a robot to intelligently predict a human companion’s intended future actions. We develop a Human Intent Prediction (HIP) system, which can model human choice, to satisfy these needs. This system, when given a time history of previous actions as input, predicts the most likely action a human agent will next make to a robot’s task scheduling system. Our HIP system is applied to an intra-vehicle activity (IVA) space robotics application. We use data from preliminary human subject testing to formulate and populate our models in an offline learning process that illustrates how the models can adapt to better predict intent as new training data is incorporated.